Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Sparse Continuous Distributions and Fenchel-Young Losses André F . T . Martins , Marcos Treviso , António Farinhas , Pedro M . Q . Aguiar , Mário A . T . Figueiredo , Mathieu Blondel , Vlad Niculae 23(257 1 74, 2022. Abstract Exponential families are widely used in machine learning , including many distributions in continuous and discrete domains e.g . Gaussian , Dirichlet , Poisson , and categorical distributions via the softmax transformation Distributions in each of these families have fixed support . In contrast , for finite domains , recent work on sparse alternatives to softmax e.g . sparsemax ,
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Network Regression with Graph Laplacians Yidong Zhou , Hans-Georg Müller 23(320 1 41, 2022. Abstract Network data are increasingly available in various research fields , motivating statistical analysis for populations of networks , where a network as a whole is viewed as a data point . The study of how a network changes as a function of covariates is often of paramount interest . However , due to the non-Euclidean nature of networks , basic statistical tools available for scalar and vector data are no longer applicable . This motivates an extension of the notion of regression to the case where responses are
Updated: 2022-12-31 14:49:52
: Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us ReservoirComputing.jl : An Efficient and Modular Library for Reservoir Computing Models Francesco Martinuzzi , Chris Rackauckas , Anas Abdelrehim , Miguel D . Mahecha , Karin Mora 23(288 1 8, 2022. Abstract We introduce ReservoirComputing.jl , an open source Julia library for reservoir computing models . It is designed for temporal or sequential tasks such as time series prediction and modeling complex dynamical systems . As such it is suited to process a range of complex spatio-temporal data sets , from mathematical models to climate data . The key ideas of reservoir computing are the model architecture ,
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Approximate Bayesian Computation via Classification Yuexi Wang , Tetsuya Kaji , Veronika Rockova 23(350 1 49, 2022. Abstract Approximate Bayesian Computation ABC enables statistical inference in simulator-based models whose likelihoods are difficult to calculate but easy to simulate from . ABC constructs a kernel-type approximation to the posterior distribution through an accept reject mechanism which compares summary statistics of real and simulated data . To obviate the need for summary statistics , we directly compare empirical distributions with a Kullback-Leibler KL divergence estimator obtained via
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Integral Autoencoder Network for Discretization-Invariant Learning Yong Zheng Ong , Zuowei Shen , Haizhao Yang 23(286 1 45, 2022. Abstract Discretization invariant learning aims at learning in the infinite-dimensional function spaces with the capacity to process heterogeneous discrete representations of functions as inputs and or outputs of a learning model . This paper proposes a novel deep learning framework based on integral autoencoders IAE-Net for discretization invariant learning . The basic building block of IAE-Net consists of an encoder and a decoder as integral transforms with data-driven kernels ,
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Hamilton-Jacobi equations on graphs with applications to semi-supervised learning and data depth Jeff Calder , Mahmood Ettehad 23(318 1 62, 2022. Abstract Shortest path graph distances are widely used in data science and machine learning , since they can approximate the underlying geodesic distance on the data manifold . However , the shortest path distance is highly sensitive to the addition of corrupted edges in the graph , either through noise or an adversarial perturbation . In this paper we study a family of Hamilton-Jacobi equations on graphs that we call the p$-eikonal equation . We show that the
Updated: 2022-12-31 14:49:52
This paper presents Deepchecks, a Python library for comprehensively validating machine learning models and data. Our goal is to provide an easy-to-use library comprising many checks related to various issues, such as model predictive performance, data integrity, data distribution mismatches, and more. The package is distributed under the GNU Affero General Public License and relies on core libraries from the scientific Python ecosystem: scikit-learn, PyTorch, NumPy, pandas, and SciPy.
Updated: 2022-12-31 14:49:52
The optimization and machine learning toolkit (OMLT) is an open-source software package incorporating neural network and gradient-boosted tree surrogate models, which have been trained using machine learning, into larger optimization problems. We discuss the advances in optimization technology that made OMLT possible and show how OMLT seamlessly integrates with the algebraic modeling language Pyomo. We demonstrate how to use OMLT for solving decision-making problems in both computer science and engineering.
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Nonparametric Neighborhood Selection in Graphical Models Hao Dong , Yuedong Wang 23(317 1 36, 2022. Abstract The neighborhood selection method directly explores the conditional dependence structure and has been widely used to construct undirected graphical models . However , except for some special cases with discrete data , there is little research on nonparametric methods for neighborhood selection with mixed data . This paper develops a fully nonparametric neighborhood selection method under a consolidated smoothing spline ANOVA SS ANOVA decomposition framework . The proposed model is flexible and contains
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Scalable Gaussian-process regression and variable selection using Vecchia approximations Jian Cao , Joseph Guinness , Marc G . Genton , Matthias Katzfuss 23(348 1 30, 2022. Abstract Gaussian process GP regression is a flexible , nonparametric approach to regression that naturally quantifies uncertainty . In many applications , the number of responses and covariates are both large , and a goal is to select covariates that are related to the response . For this setting , we propose a novel , scalable algorithm , coined VGPR , which optimizes a penalized GP log-likelihood based on the Vecchia GP approximation ,
Updated: 2022-12-31 14:49:52
In this paper, we introduce d3rlpy, an open-sourced offline deep reinforcement learning (RL) library for Python. d3rlpy supports a set of offline deep RL algorithms as well as off-policy online algorithms via a fully documented plug-and-play API. To address a reproducibility issue, we conduct a large-scale benchmark with D4RL and Atari 2600 dataset to ensure implementation quality and provide experimental scripts and full tables of results. The d3rlpy source code can be found on GitHub: https://github.com/takuseno/d3rlpy.
Updated: 2022-12-31 14:49:52
We study the minimax rates of the label shift problem in non-parametric classification. In addition to the unsupervised setting in which the learner only has access to unlabeled examples from the target domain, we also consider the setting in which a small number of labeled examples from the target domain is available to the learner. Our study reveals a difference in the difficulty of the label shift problem in the two settings, and we attribute this difference to the availability of data from the target domain to estimate the class conditional distributions in the latter setting. We also show that a class proportion estimation approach is minimax rate-optimal in the unsupervised setting.
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Constraint Reasoning Embedded Structured Prediction Nan Jiang , Maosen Zhang , Willem-Jan van Hoeve , Yexiang Xue 23(345 1 40, 2022. Abstract Many real-world structured prediction problems need machine learning to capture data distribution and constraint reasoning to ensure structure validity . Nevertheless , constrained structured prediction is still limited in real-world applications because of the lack of tools to bridge constraint satisfaction and machine learning . In this paper , we propose COnstraint REasoning embedded Structured Prediction Core-Sp a scalable constraint reasoning and machine learning
Updated: 2022-12-31 14:49:52
It has long been thought that high-dimensional data encountered in many practical machine learning tasks have low-dimensional structure, i.e., the manifold hypothesis holds. A natural question, thus, is to estimate the intrinsic dimension of a given population distribution from a finite sample. We introduce a new estimator of the intrinsic dimension and provide finite sample, non-asymptotic guarantees. We then apply our techniques to get new sample complexity bounds for Generative Adversarial Networks (GANs) depending only on the intrinsic dimension of the data.
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Jump Gaussian Process Model for Estimating Piecewise Continuous Regression Functions Chiwoo Park 23(278 1 37, 2022. Abstract This paper presents a Gaussian process GP model for estimating piecewise continuous regression functions . In many scientific and engineering applications of regression analysis , the underlying regression functions are often piecewise continuous in that data follow different continuous regression models for different input regions with discontinuities across regions . However , many conventional GP regression approaches are not designed for piecewise regression analysis . There are
Updated: 2022-12-31 14:49:52
: Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Policy Gradient and Actor-Critic Learning in Continuous Time and Space : Theory and Algorithms Yanwei Jia , Xun Yu Zhou 23(275 1 50, 2022. Abstract We study policy gradient PG for reinforcement learning in continuous time and space under the regularized exploratory formulation developed by Wang et al . 2020 We represent the gradient of the value function with respect to a given parameterized stochastic policy as the expected integration of an auxiliary running reward function that can be evaluated using samples and the current value function . This representation effectively turns PG into a policy evaluation
Updated: 2022-12-31 14:49:52
We propose and analyse a reduced-rank method for solving least-squares regression problems with infinite dimensional output. We derive learning bounds for our method, and study under which setting statistical performance is improved in comparison to full-rank method. Our analysis extends the interest of reduced-rank regression beyond the standard low-rank setting to more general output regularity assumptions. We illustrate our theoretical insights on synthetic least-squares problems. Then, we propose a surrogate structured prediction method derived from this reduced-rank method. We assess its benefits on three different problems: image reconstruction, multi-label classification, and metabolite identification.
Updated: 2022-12-31 14:49:52
This paper focuses on learning rate analysis of Nystrom regularization with sequential sub-sampling for $\tau$-mixing time series. Using a recently developed Banach-valued Bernstein inequality for $\tau$-mixing sequences and an integral operator approach based on second-order decomposition, we succeed in deriving almost optimal learning rates of Nystrom regularization with sequential sub-sampling for $\tau$-mixing time series. A series of numerical experiments are carried out to verify our theoretical results, showing the excellent learning performance of Nystrom regularization with sequential sub-sampling in learning massive time series data. All these results extend the applicable range of Nystr\"{o}m regularization from i.i.d. samples to non-i.i.d. sequences.
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Communication-Constrained Distributed Quantile Regression with Optimal Statistical Guarantees Kean Ming Tan , Heather Battey , Wen-Xin Zhou 23(272 1 61, 2022. Abstract We address the problem of how to achieve optimal inference in distributed quantile regression without stringent scaling conditions . This is challenging due to the non-smooth nature of the quantile regression QR loss function , which invalidates the use of existing methodology . The difficulties are resolved through a double-smoothing approach that is applied to the local at each data source and global objective functions . Despite the reliance
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Fast Stagewise Sparse Factor Regression Kun Chen , Ruipeng Dong , Wanwan Xu , Zemin Zheng 23(271 1 45, 2022. Abstract Sparse factorization of a large matrix is fundamental in modern statistical learning . In particular , the sparse singular value decomposition has been utilized in many multivariate regression methods . The appeal of this factorization is owing to its power in discovering a highly-interpretable latent association network . However , many existing methods are either ad hoc without a general performance guarantee , or are computationally intensive . We formulate the statistical problem as a
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Toward Understanding Convolutional Neural Networks from Volterra Convolution Perspective Tenghui Li , Guoxu Zhou , Yuning Qiu , Qibin Zhao 23(311 1 50, 2022. Abstract We make an attempt to understand convolutional neural network by exploring the relationship between deep convolutional neural networks and Volterra convolutions . We propose a novel approach to explain and study the overall characteristics of neural networks without being disturbed by the horribly complex architectures . Specifically , we attempt to convert the basic structures of a convolutional neural network CNN and their combinations to the
Updated: 2022-12-31 14:49:52
: Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us SGD with Coordinate Sampling : Theory and Practice Rémi Leluc , François Portier 23(342 1 47, 2022. Abstract While classical forms of stochastic gradient descent algorithm treat the different coordinates in the same way , a framework allowing for adaptive non uniform coordinate sampling is developed to leverage structure in data . In a non-convex setting and including zeroth-order gradient estimate , almost sure convergence as well as non-asymptotic bounds are established . Within the proposed framework , we develop an algorithm , MUSKETEER , based on a reinforcement strategy : after collecting information
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Detecting Latent Communities in Network Formation Models Shujie Ma , Liangjun Su , Yichong Zhang 23(310 1 61, 2022. Abstract This paper proposes a logistic undirected network formation model which allows for assortative matching on observed individual characteristics and the presence of edge-wise fixed effects . We model the coefficients of observed characteristics to have a latent community structure and the edge-wise fixed effects to be of low rank . We propose a multi-step estimation procedure involving nuclear norm regularization , sample splitting , iterative logistic regression and spectral clustering to
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Learning linear non-Gaussian directed acyclic graph with diverging number of nodes Ruixuan Zhao , Xin He , Junhui Wang 23(269 1 34, 2022. Abstract An acyclic model , often depicted as a directed acyclic graph DAG has been widely employed to represent directional causal relations among collected nodes . In this article , we propose an efficient method to learn linear non-Gaussian DAG in high dimensional cases , where the noises can be of any continuous non-Gaussian distribution . The proposed method leverages the concept of topological layer to facilitate the DAG learning , and its theoretical justification in
Updated: 2022-12-31 14:49:52
: Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Information-theoretic Classification Accuracy : A Criterion that Guides Data-driven Combination of Ambiguous Outcome Labels in Multi-class Classification Chihao Zhang , Yiling Elaine Chen , Shihua Zhang , Jingyi Jessica Li 23(341 1 65, 2022. Abstract Outcome labeling ambiguity and subjectivity are ubiquitous in real-world datasets . While practitioners commonly combine ambiguous outcome labels for all data points instances in an ad hoc way to improve the accuracy of multi-class classification , there lacks a principled approach to guide the label combination for all data points by any optimality criterion .
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us A Computationally Efficient Framework for Vector Representation of Persistence Diagrams Kit C Chan , Umar Islambekov , Alexey Luchinsky , Rebecca Sanders 23(268 1 33, 2022. Abstract In Topological Data Analysis , a common way of quantifying the shape of data is to use a persistence diagram PD PDs are multisets of points in R^2$ computed using tools of algebraic topology . However , this multi-set structure limits the utility of PDs in applications . Therefore , in recent years efforts have been directed towards extracting informative and efficient summaries from PDs to broaden the scope of their use for
Updated: 2022-12-31 14:49:52
In this paper, we present Tianshou, a highly modularized Python library for deep reinforcement learning (DRL) that uses PyTorch as its backend. Tianshou intends to be research-friendly by providing a flexible and reliable infrastructure of DRL algorithms. It supports online and offline training with more than 20 classic algorithms through a unified interface. To facilitate related research and prove Tianshou's reliability, we have released Tianshou's benchmark of MuJoCo environments, covering eight classic algorithms with state-of-the-art performance. We open-sourced Tianshou at https://github.com/thu-ml/tianshou/.
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Functional Linear Regression with Mixed Predictors Daren Wang , Zifeng Zhao , Yi Yu , Rebecca Willett 23(266 1 94, 2022. Abstract We study a functional linear regression model that deals with functional responses and allows for both functional covariates and high-dimensional vector covariates . The proposed model is flexible and nests several functional regression models in the literature as special cases . Based on the theory of reproducing kernel Hilbert spaces RKHS we propose a penalized least squares estimator that can accommodate functional variables observed on discrete sample points . Besides a
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Fundamental Limits and Tradeoffs in Invariant Representation Learning Han Zhao , Chen Dan , Bryon Aragam , Tommi S . Jaakkola , Geoffrey J . Gordon , Pradeep Ravikumar 23(340 1 49, 2022. Abstract A wide range of machine learning applications such as privacy-preserving learning , algorithmic fairness , and domain adaptation generalization among others , involve learning invariant representations of the data that aim to achieve two competing goals : a maximize information or accuracy with respect to a target response , and b maximize invariance or independence with respect to a set of protected features e.g .
Updated: 2022-12-31 14:49:52
: Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us On Regularized Square-root Regression Problems : Distributionally Robust Interpretation and Fast Computations Hong T.M . Chu , Kim-Chuan Toh , Yangjing Zhang 23(308 1 39, 2022. Abstract Square-root loss regularized models have recently become popular in linear regression due to their nice statistical properties . Moreover , some of these models can be interpreted as the distributionally robust optimization counterparts of the traditional least-squares regularized models . In this paper , we give a unified proof to show that any square-root regularized model whose penalty function being the sum of a simple
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us A Random Matrix Perspective on Random Tensors José Henrique de M . Goulart , Romain Couillet , Pierre Comon 23(264 1 36, 2022. Abstract Several machine learning problems such as latent variable model learning and community detection can be addressed by estimating a low-rank signal from a noisy tensor . Despite recent substantial progress on the fundamental limits of the corresponding estimators in the large-dimensional setting , some of the most significant results are based on spin glass theory , which is not easily accessible to non-experts . We propose a sharply distinct and more elementary approach ,
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Learning Temporal Evolution of Spatial Dependence with Generalized Spatiotemporal Gaussian Process Models Shiwei Lan 23(259 1 53, 2022. Abstract A large number of scientific studies involve high-dimensional spatiotemporal data with complicated relationships . In this paper , we focus on a type of space-time interaction named temporal evolution of spatial dependence TESD which is a zero time-lag spatiotemporal covariance . For this purpose , we propose a novel Bayesian nonparametric method based on non-stationary spatiotemporal Gaussian process STGP The classic STGP has a covariance kernel separable in space
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Learning from Noisy Pairwise Similarity and Unlabeled Data Songhua Wu , Tongliang Liu , Bo Han , Jun Yu , Gang Niu , Masashi Sugiyama 23(307 1 34, 2022. Abstract SU classification employs similar S data pairs two examples belong to the same class and unlabeled U data points to build a classifier , which can serve as an alternative to the standard supervised trained classifiers requiring data points with class labels . SU classification is advantageous because in the era of big data , more attention has been paid to data privacy . Datasets with specific class labels are often difficult to obtain in real-world
Updated: 2022-12-31 14:49:52
This paper presents a new approach for regression tree-based models, such as simple regression tree, random forest and gradient boosting, in settings involving correlated data. We show the problems that arise when implementing standard regression tree-based models, which ignore the correlation structure. Our new approach explicitly takes the correlation structure into account in the splitting criterion, stopping rules and fitted values in the leaves, which induces some major modifications of standard methodology. The superiority of our new approach over tree-based models that do not account for the correlation, and over previous work that integrated some aspects of our approach, is supported by simulation experiments and real data analyses.
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Statistical Optimality and Computational Efficiency of Nystrom Kernel PCA Nicholas Sterge , Bharath K . Sriperumbudur 23(337 1 32, 2022. Abstract Kernel methods provide an elegant framework for developing nonlinear learning algorithms from simple linear methods . Though these methods have superior empirical performance in several real data applications , their usefulness is inhibited by the significant computational burden incurred in large sample situations . Various approximation schemes have been proposed in the literature to alleviate these computational issues , and the approximate kernel machines are
Updated: 2022-12-31 14:49:52
: Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Tree-Values : Selective Inference for Regression Trees Anna C . Neufeld , Lucy L . Gao , Daniela M . Witten 23(305 1 43, 2022. Abstract We consider conducting inference on the output of the Classification and Regression Tree CART Breiman et al . 1984 algorithm . A naive approach to inference that does not account for the fact that the tree was estimated from the data will not achieve standard guarantees , such as Type 1 error rate control and nominal coverage . Thus , we propose a selective inference framework for conducting inference on a fitted CART tree . In a nutshell , we condition on the fact that the
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Faster Randomized Interior Point Methods for Tall Wide Linear Programs Agniva Chowdhury , Gregory Dexter , Palma London , Haim Avron , Petros Drineas 23(336 1 48, 2022. Abstract Linear programming LP is an extremely useful tool which has been successfully applied to solve various problems in a wide range of areas , including operations research , engineering , economics , or even more abstract mathematical areas such as combinatorics . It is also used in many machine learning applications , such as ell_1$-regularized SVMs , basis pursuit , nonnegative matrix factorization , etc . Interior Point Methods IPMs
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Variational Inference in high-dimensional linear regression Sumit Mukherjee , Subhabrata Sen 23(304 1 56, 2022. Abstract We study high-dimensional bayesian linear regression with product priors . Using the nascent theory of non-linear large deviations Chatterjee and Dembo , 2016 we derive sufficient conditions for the leading-order correctness of the naive mean-field approximation to the log-normalizing constant of the posterior distribution . Subsequently , assuming a true linear model for the observed data , we derive a limiting infinite dimensional variational formula for the log normalizing constant for
Updated: 2022-12-31 14:49:52
: Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Causal Aggregation : Estimation and Inference of Causal Effects by Constraint-Based Data Fusion Jaime Roquero Gimenez , Dominik Rothenhäusler 23(335 1 60, 2022. Abstract In causal inference , it is common to estimate the causal effect of a single treatment variable on an outcome . However , practitioners may also be interested in the effect of simultaneous interventions on multiple covariates of a fixed target variable . We propose a novel method that allows to estimate the effect of joint interventions using data from different experiments in which only very few variables are manipulated . If there is only
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Graph Partitioning and Sparse Matrix Ordering using Reinforcement Learning and Graph Neural Networks Alice Gatti , Zhixiong Hu , Tess Smidt , Esmond G . Ng , Pieter Ghysels 23(303 1 28, 2022. Abstract We present a novel method for graph partitioning , based on reinforcement learning and graph convolutional neural networks . Our approach is to recursively partition coarser representations of a given graph . The neural network is implemented using SAGE graph convolution layers , and trained using an advantage actor critic A2C agent . We present two variants , one for finding ean edge separator that minimizes the
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us On Instrumental Variable Regression for Deep Offline Policy Evaluation Yutian Chen , Liyuan Xu , Caglar Gulcehre , Tom Le Paine , Arthur Gretton , Nando de Freitas , Arnaud Doucet 23(302 1 40, 2022. Abstract We show that the popular reinforcement learning RL strategy of estimating the state-action value Q-function by minimizing the mean squared Bellman error leads to a regression problem with confounding , the inputs and output noise being correlated . Hence , direct minimization of the Bellman error can result in significantly biased Q-function estimates . We explain why fixing the target Q-network in Deep
Updated: 2022-12-31 14:49:52
: Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Distributional Random Forests : Heterogeneity Adjustment and Multivariate Distributional Regression Domagoj Cevid , Loris Michel , Jeffrey Näf , Peter Bühlmann , Nicolai Meinshausen 23(333 1 79, 2022. Abstract Random Forest is a successful and widely used regression and classification algorithm . Part of its appeal and reason for its versatility is its implicit construction of a kernel-type weighting function on training data , which can also be used for targets other than the original mean estimation . We propose a novel forest construction for multivariate responses based on their joint conditional
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Theoretical Foundations of t-SNE for Visualizing High-Dimensional Clustered Data T . Tony Cai , Rong Ma 23(301 1 54, 2022. Abstract This paper investigates the theoretical foundations of the t-distributed stochastic neighbor embedding t-SNE algorithm , a popular nonlinear dimension reduction and data visualization method . A novel theoretical framework for the analysis of t-SNE based on the gradient descent approach is presented . For the early exaggeration stage of t-SNE , we show its asymptotic equivalence to power iterations based on the underlying graph Laplacian , characterize its limiting behavior , and
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Maximum sampled conditional likelihood for informative subsampling HaiYing Wang , Jae Kwang Kim 23(332 1 50, 2022. Abstract Subsampling is a computationally effective approach to extract information from massive data sets when computing resources are limited . After a subsample is taken from the full data , most available methods use an inverse probability weighted IPW objective function to estimate the model parameters . The IPW estimator does not fully utilize the information in the selected subsample . In this paper , we propose to use the maximum sampled conditional likelihood estimator MSCLE based on the
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us More Powerful Conditional Selective Inference for Generalized Lasso by Parametric Programming Vo Nguyen Le Duy , Ichiro Takeuchi 23(300 1 37, 2022. Abstract Conditional selective inference SI has been studied intensively as a new statistical inference framework for data-driven hypotheses . The basic concept of conditional SI is to make the inference conditional on the selection event , which enables an exact and valid statistical inference to be conducted even when the hypothesis is selected based on the data . Conditional SI has mainly been studied in the context of model selection , such as vanilla lasso or
Updated: 2022-12-31 14:49:52
, Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us The Geometry of Uniqueness , Sparsity and Clustering in Penalized Estimation Ulrike Schneider , Patrick Tardivel 23(331 1 36, 2022. Abstract We provide a necessary and sufficient condition for the uniqueness of penalized least-squares estimators whose penalty term is given by a norm with a polytope unit ball , covering a wide range of methods including SLOPE , PACS , fused , clustered and classical LASSO as well as the related method of basis pursuit . We consider a strong type of uniqueness that is relevant for statistical problems . The uniqueness condition is geometric and involves how the row span of the
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Interpretable Classification of Categorical Time Series Using the Spectral Envelope and Optimal Scalings Zeda Li , Scott A . Bruce , Tian Cai 23(299 1 31, 2022. Abstract This article introduces a novel approach to the classification of categorical time series under the supervised learning paradigm . To construct meaningful features for categorical time series classification , we consider two relevant quantities : the spectral envelope and its corresponding set of optimal scalings . These quantities characterize oscillatory patterns in a categorical time series as the largest possible power at each frequency ,
Updated: 2022-12-31 14:49:52
: Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us ALMA : Alternating Minimization Algorithm for Clustering Mixture Multilayer Network Xing Fan , Marianna Pensky , Feng Yu , Teng Zhang 23(330 1 46, 2022. Abstract The paper considers a Mixture Multilayer Stochastic Block Model MMLSBM where layers can be partitioned into groups of similar networks , and networks in each group are equipped with a distinct Stochastic Block Model . The goal is to partition the multilayer network into clusters of similar layers , and to identify communities in those layers . Jing et al . 2020 introduced the MMLSBM and developed a clustering methodology , TWIST , based on
Updated: 2022-12-31 14:49:52
: Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us JsonGrinder.jl : automated differentiable neural architecture for embedding arbitrary JSON data Šimon Mandlík , Matěj Račinský , Viliam Lisý , Tomáš Pevný 23(298 1 5, 2022. Abstract Standard machine learning ML problems are formulated on data converted into a suitable tensor representation . However , there are data sources , for example in cybersecurity , that are naturally represented in a unifying hierarchical structure , such as XML , JSON , and Protocol Buffers . Converting this data to a tensor representation is usually done by manual feature engineering , which is laborious , lossy , and prone to bias
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Joint Continuous and Discrete Model Selection via Submodularity Jonathan Bunton , Paulo Tabuada 23(329 1 42, 2022. Abstract In model selection problems for machine learning , the desire for a well-performing model with meaningful structure is typically expressed through a regularized optimization problem . In many scenarios , however , the meaningful structure is specified in some discrete space , leading to difficult nonconvex optimization problems . In this paper , we connect the model selection problem with structure-promoting regularizers to submodular function minimization with continuous and discrete
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Stable Classification Dimitris Bertsimas , Jack Dunn , Ivan Paskov 23(296 1 53, 2022. Abstract We address the problem of instability of classification models : small changes in the training data leading to large changes in the resulting model and predictions . This phenomenon is especially well established for single tree based methods such as CART , however it is present in all classification methods . We apply robust optimization to improve the stability of four of the most commonly used classification methods : Random Forests , Logistic Regression , Support Vector Machines , and Optimal Classification Trees
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Semiparametric Inference For Causal Effects In Graphical Models With Hidden Variables Rohit Bhattacharya , Razieh Nabi , Ilya Shpitser 23(295 1 76, 2022. Abstract Identification theory for causal effects in causal models associated with hidden variable directed acyclic graphs DAGs is well studied . However , the corresponding algorithms are underused due to the complexity of estimating the identifying functionals they output . In this work , we bridge the gap between identification and estimation of population-level causal effects involving a single treatment and a single outcome . We derive influence function
Updated: 2022-12-31 14:49:52
: Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Project and Forget : Solving Large-Scale Metric Constrained Problems Rishi Sonthalia , Anna C . Gilbert 23(326 1 54, 2022. Abstract Many important machine learning problems can be formulated as highly constrained convex optimization problems . One important example is metric constrained problems . In this paper , we show that standard optimization techniques can not be used to solve metric constrained problem . To solve such problems , we provide a general active set framework , called Project and Forget , and several variants thereof that use Bregman projections . Project and Forget is a general purpose
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Linearization and Identification of Multiple-Attractor Dynamical Systems through Laplacian Eigenmaps Bernardo Fichera , Aude Billard 23(294 1 35, 2022. Abstract Dynamical Systems DS are fundamental to the modeling and understanding time evolving phenomena , and have application in physics , biology and control . As determining an analytical description of the dynamics is often difficult , data-driven approaches are preferred for identifying and controlling nonlinear DS with multiple equilibrium points . Identification of such DS has been treated largely as a supervised learning problem . Instead , we focus on
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Improving Bayesian Network Structure Learning in the Presence of Measurement Error Yang Liu , Anthony C . Constantinou , Zhigao Guo 23(324 1 28, 2022. Abstract Structure learning algorithms that learn the graph of a Bayesian network from observational data often do so by assuming the data correctly reflect the true distribution of the variables . However , this assumption does not hold in the presence of measurement error , which can lead to spurious edges . This is one of the reasons why the synthetic performance of these algorithms often overestimates real-world performance . This paper describes a heuristic
Updated: 2022-12-31 14:49:52
: Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Generalized Matrix Factorization : efficient algorithms for fitting generalized linear latent variable models to large data arrays Lukasz Kidzinski , Francis K.C . Hui , David I . Warton , Trevor J . Hastie 23(291 1 29, 2022. Abstract Unmeasured or latent variables are often the cause of correlations between multivariate measurements , which are studied in a variety of fields such as psychology , ecology , and medicine . For Gaussian measurements , there are classical tools such as factor analysis or principal component analysis with a well-established theory and fast algorithms . Generalized Linear Latent
Updated: 2022-12-31 14:49:52
We show that a simple community detection algorithm originated from stochastic blockmodel literature achieves consistency, and even optimality, for a broad and flexible class of sparse latent space models. The class of models includes latent eigenmodels (Hoff, 2008). The community detection algorithm is based on spectral clustering followed by local refinement via normalized edge counting. It is easy to implement and attains high accuracy with a low computational budget. The proof of its optimality depends on a neat equivalence between likelihood ratio test and edge counting in a simple vs. simple hypothesis testing problem that underpins the refinement step, which could be of independent interest.
Updated: 2022-12-31 14:49:52
Home Page Papers Submissions News Editorial Board Special Issues Open Source Software Proceedings PMLR Transactions TMLR Search Statistics Login Frequently Asked Questions Contact Us Regularized and Smooth Double Core Tensor Factorization for Heterogeneous Data Davoud Ataee Tarzanagh , George Michailidis 23(290 1 49, 2022. Abstract We introduce a general tensor model suitable for data analytic tasks for heterogeneous datasets , wherein there are joint low-rank structures within groups of observations , but also discriminative structures across different groups . To capture such complex structures , a double core tensor DCOT factorization model is introduced together with a family of smoothing loss functions . By leveraging the proposed smoothing function , the model accurately estimates
